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REAL PARTY IN INTEREST 

The real party in interest in this application is Hewlett-Packard 
Development Company, L.P. 

RELATED APPEALS AND INTERFERENCES 

Appellant is unaware of any other related appeals or interferences that 
may directly affect or be directly affected by or have a bearing on the Board's 
decision in this appeal. 

STATUS OF THE CLAIMS 

Claims 1-45 stand rejected under 35 U.S.C. §103(a) as being 
unpatentable over U.S. Patent Application 2003/0001890 of Brin {"Brin") and 
U.S. Patent No. 5,686,957 of Baker ("Baker") and U.S. Patent Application 
2004/0172252 of Aoki et al {"Aoki"). 

Appellant appeals the rejection of all of the pending claims 1-45. Claims 
1-45 as currently pending are set forth in the attached Appendix. 

STATUS OF AMENDMENTS 

Appellant is unaware of any amendments filed after the Final Office 
Action mailed 7-11-07 which finally rejected claims 1-45. 
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SUMMARY OF CLAIMED SUBJECT MATTER 

Independent claims 1, 22, and 34 are directed to enabling multiple 
communications interactions, e.g. multiple side groups, to form among 
multiple individuals that are physically present in different environments by 
using respective sets of sensing and rendering components in the 
environments to detect the communication interactions and then capturing 
and combining the media data for the sensing and rendering components in 
response to the activities of the individuals associated with the respective 
communication interactions. (See pages 1-2 of Appellant 7 s specification). 

Independent claim 1 is a system for communication including a first set 
of sensing and rendering components arranged to cover physical movements 
of multiple individuals present in a first environment (See Figure 5, element 
250 and pages 15-16 of Appellant's specification) and a second set of sensing 
and rendering components arranged to cover physical movements of 
multiple individuals present in a second environment (See Figure 5, element 
252 and pages 15-16 of Appellant's specification). A system according to claim 
1 includes an interest thread detector (See Figure 1, element 16 and page 5 of 
Appellant's specification) that uses the first and second set of sensing and 
rendering components to detect multiple communication interactions each 
involving a respective subset of the individuals present in the first and second 
environments (See Figure 1, element 16 and page 5, last paragraph of 
Appellant 7 s specification) and that maintains an interest thread for each 
communication interaction (See Figure 1, element 16 and page 5, last 
paragraph of Appellant's specification) and further includes a communication 
provider (See Figure 1, element 18 and page 6, first paragraph of Appellant 7 s 
specification) that captures a set of media data from the sensing components 
and that combines the captured media data in response to the respective 
activities of each subset of the individuals (See Figure 2, element 36 and page 
6, fourth paragraph of Appellant's specification) and that communicates the 
combined media data to the rendering components (See Figure 2, element 38 
and page 6, fourth paragraph of Appellant's specification). 

Independent claim 22 is directed to a method for communication that 
includes providing a first set of sensing and rendering components for 
covering physical movements of multiple individuals present in a first 

Appeal Brief 4 

Application Ser. No.: 10/682,542 Attorney Docket No.: 200315123-1 



environment (See Figure 5, element 250 and pages 15-16 of Appellant's 
specification) and providing a second set of sensing and rendering 
components for covering physical movements of multiple individuals present 
in a second environment (See Figure 5, element 252 and pages 15-16 of 
Appellant's specification). A method according to claim 22 further includes 
detecting multiple communication interactions each a communication 
interaction involving a respective subset of the individuals present in the first 
and second environments (See Figure 2, element 30 and page 6, second 
paragraph of Appellant 7 s specification) and maintaining an interest thread for 
the each detected communication interaction (See Figure 2, element 32 and 
page 6, second paragraph of Appellant's specification) and capturing a set of 
media data from the sensing components (See Figure 2, element 34 and page 
6, fourth paragraph of Appellant's specification) and combining the captured 
media data in response to the respective activities of the respective subset of 
the individuals (See Figure 2, element 36 and page 6, fourth paragraph of 
Appellant s specification) and communicating the combined media data to the 
rendering components (See Figure 2, element 38 and page 6, fourth 
paragraph of Appellant's specification). 

Independent claim 34 is directed to a computer-readable storage media 
that includes providing a first set of sensing and rendering components for 
covering physical movements of multiple individuals present in a first 
environment (See Figure 5, element 250 and pages 15-16 of Appellant's 
specification) and providing a second set of sensing and rendering 
components for covering physical movements of multiple individuals present 
in a second environment (See Figure 5, element 252 and pages 15-16 of 
Appellant 7 s specification). A method according to claim 22 further includes 
detecting multiple communication interactions each a communication 
interaction involving a respective subset of the individuals present in the first 
and second environments (See Figure 2, element 30 and page 6, second 
paragraph of Appellant's specification) and maintaining an interest thread for 
the each detected communication interaction (See Figure 2, element 32 and 
page 6, second paragraph of Appellant's specification) and capturing a set of 
media data from the sensing components (See Figure 2, element 34 and page 
6, fourth paragraph of Appellant's specification) and combining the captured 
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media data in response to the respective activities of the respective subset of 
the individuals (See Figure 2, element 36 and page 6, fourth paragraph of 
Appellant" s specification) and communicating the combined media data to the 
rendering components (See Figure 2, element 38 and page 6, fourth 
paragraph of Appellant's specification). 
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GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 



I: Rejection of claims 1-45 as being obvious in view of Brin 
and Baker and Aoki. 
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ARGUMENT 



I: Claims 1-45 are not obvious in view of Brin and Baker and 
Aoki because Brin and Baker and Aoki do not disclose or 
suggest the limitations of independent claims 1, 22, and 34. 

Appellant respectfully submits that independent claims 1, 22 and 34, 
and claims 2-21, 23-33, and 35-45 which depend from independent claims 1, 22 
and 34, are not obvious in view of Brin and Baker and Aoki because Brin and 
Baker and Aoki do not disclose or suggest using respective sets of sensing and 
rendering components to detect multiple communication interactions among 
multiple individuals that are physically present in a first and a second 
environment as claimed in independent claims 1, 22 and 34. Appellant also 
submits that Brin and Baker and Aoki do not disclose or suggest capturing and 
combining media data in response to the activities of the individuals 
associated with the respective detected communication interactions as claimed 
in independent claims 1, 22 and 34. 
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A: Brin and Baker and Aoki do not disclose or suggest 
using respective sets of sensing and rendering components 
to detect multiple communication interactions among 
multiple individuals physically present a first and a second 
environment as claimed in independent claims 1, 22, and 34. 

Appellant respectfully submits that Brin and Baker and Aoki do not 
disclose or suggest using respective sets of sensing and rendering 
components to detect multiple communication interactions among multiple 
individuals that are physically present in a first and a second environment as 
claimed in claims 1, 22 and 34. Instead, Brin discloses an online chat system 
that uses movements of icons inside of computer system display windows to 
detect communication interactions {Brin, Figures 1-16 and paragraphs 82-120) 
and Baker discloses steering a video camera toward an individual that is 
speaking {Baker, Abstract and col. 5, lines 32-37) and Aoki discloses identifying 
a conversation among a set of individuals positioned at their computers {Aoki, 
Figure 1 and paragraph 11). 

The examiner has stated that Brin and Baker taken together disclose 
using sensing and rendering components to detect multiple communication 
interactions among multiple individuals present in a first and a second 
environment because the system of Brin enables users to participate in 
multiple "real-world" conversations 1 (page 2, Office Action, 7-11-07) and 
because Baker discloses sensing and rendering components for covering the 
physical movements of multiple individuals that are present in an 
environment (page 3, Office Action, 7-11-07). 

It is respectfully submitted that the examiner has lifted the term "real- 
world" from Brin in attempt to show that the teachings of Brin are applicable 
to communication interactions that occur in among individuals that are 
physically present in a first and a second environment as claimed in claims 1, 
22 and 34. In truth, Brin only uses the term "real-world" to denote those 
conversational characteristics that occur normally in the physical world. {Brin, 
paragraph 11). The online chat system of Brin itself does not detect those 
conversational characteristics in a physical environment but instead uses a 
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computer program to emulate those conversational characteristics using 
moveable icons in a window on a display screen of a computer terminal. 
(Brin, paragraphs 12 and 82). Paragraphs 1-250 of Brin clearly describe an 
online chat system in which users seated at computer terminals are 
represented as icons in a window on a display screen and in which 
communication interactions are detected by detecting the movements of the 
icons within the window. 

It is further submitted that one of skill in the art would not combine 
Baker with Brin as suggested by the examiner because the teaching in Baker of 
steering a video camera toward a physical location of an individual is 
incompatible with the online chat system of Brin. It is submitted that in an 
online virtual chat system as taught by Brin determining the physical location 
of an individual as taught by Baker is irrelevant. It is submitted that the 
examiner is clearly engaging in an impermissible hindsight reconstruction of 
appellant's invention by attempting to throw together incompatible pieces of 
prior art. 



1 The examiner has actually stated that Brin enables users to perceive and participate in "a 
multiple real-world conversational characteristics." It is submitted that the examiner 
means multiple real-world conversations. 
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B: Brin and Baker and Aoki do not disclose or suggest 
capturing and combining media data in response to the 
activities of the individuals associated with the respective 
detected communication interactions as claimed in 
independent claims 1, 22, and 34. 

Appellant respectfully submits that Brin and Baker and Aoki do not 
disclose or suggest capturing and combining media data in response to the 
activities of the individuals associated with the respective detected 
communication interactions as claimed in claims 1, 22 and 34. This follows 
from the fact that Brin and Baker and Aoki do not disclose or suggest detecting 
multiple communication interactions as claimed in claims 1, 22 and 34 and so 
Brin and Baker and Aoki cannot respond to the activities of the individuals 
involved in the detected communication interactions. 
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CONCLUSION 

Appellant respectfully submits that the stated rejections cannot be 
maintained in view of the arguments set forth above. Appellant respectfully 
submits that all of the claims 1-45 are patentable under 35 U.S.C. §103 over 
the references cited by the Examiner and requests that the Board of Patent 
Appeals and Interferences direct allowance of the rejected claims. 

Respectfully submitted, 

By 

Date: p^^AL^L? 

Paul H. Horstmann 
Reg. No. 36,167 
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CLAIMS APPENDIX 



1. A system for communication, comprising: 

first set of sensing and rendering components arranged to cover 
physical movements of multiple individuals present in a first environment; 

second set of sensing and rendering components arranged to cover 
physical movements of multiple individuals present in a second environment; 

interest thread detector that uses the first and second set of sensing 
and rendering components to detect multiple communication interactions 
each involving a respective subset of the individuals present in the first and 
second environments and that maintains an interest thread for each 
communication interaction; 

communication provider that captures a set of media data from the 
sensing components and that combines the captured media data in response 
to the respective activities of each subset of the individuals and that 
communicates the combined media data to the rendering components. 

2. The system of claim 1, wherein the communication provider selects a 
respective subset of the first and second set of sensing and rendering 
components for use for each interest thread. 

3. The system of claim 1, wherein the respective activities include speech 
levels of the individuals involved in the respective interest thread. 

4. The system of claim 1, wherein the respective activities include 
gestures by the individuals involved in the respective interest thread. 

5. The system of claim 1, wherein the respective activities include 
movements by the individuals involved in the respective interest thread. 

6. The system of claim 1, wherein the respective activities include 
locations of the individuals involved in the respective interest thread. 
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7. The system of claim 1, wherein the communication provider refines 
the media data obtained from the sensor components in response to the 
respective activities. 

8. The system of claim 1, wherein the communication provider stores the 
combined media data to provide a history of each communication interaction. 

9. The system of claim 1, wherein the communication interactions include 
a communication interaction pertains to an artifact in one of the 
environments. 

10. The system of claim 9, wherein the artifact changes over time. 

11. The system of claim 9, wherein the artifact is a shared virtual writing 
surface. 

12. The system of claim 10, wherein a change to the artifact is made by one 
of the individuals involved in the interest threads. 

13. The system of claim 10, wherein the communication provider records a 
history of the artifact over time. 

14. The system of claim 1, wherein the interest thread detector detects one 
or more activities in the environments and creates an interest area for the 
detected activity. 

15. The system of claim 14, wherein the interest thread detector associates 
the interest area with another interest thread. 

16. The system of claim 1, wherein the communication interactions include 
a communication interaction that involves two or more of the individuals in 
one of the environments. 



Appeal Brief 

Application Ser. No.: 10/682,542 



14 



Attorney Docket No.: 200315123-1 



17. The system of claim 1, wherein the communication interactions include 
a communication interaction that involves one or more of the individuals in 
two of the environments. 

18. The system of claim 1, wherein the interest thread detector detects 
formation by detecting a movement of one of the individuals. 

19. The system of claim 18, wherein the movement pertains to one of the 
rendering devices. 

20. The system of claim 18, wherein the movement pertains to one of the 
other individuals. 

21. The system of claim 1, wherein one or more of the individuals is in a 
remote location and in possession of a remote sensing and rendering 
component. 

22. A method for communication, comprising: 

providing a first set of sensing and rendering components for covering 
physical movements of multiple individuals present in a first environment; 

providing a second set of sensing and rendering components for 
covering physical movements of multiple individuals present in a second 
environment; 

detecting multiple communication interactions each a communication 
interaction involving a respective subset of the individuals present in the first 
and second environments; 

maintaining an interest thread for the each detected communication 
interaction; 

capturing a set of media data from the sensing components; 

combining the captured media data in response to the respective 
activities of the respective subset of the individuals; 

communicating the combined media data to the rendering 
components. 
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23. The method of claim 22, further comprising selecting a respective 
subset of the sensing and rendering components for use for each interest 
thread. 

24. The method of claim 22, wherein combining the captured media data 
includes detecting speech levels of the corresponding individuals. 

25. The method of claim 22, wherein combining the captured media data 
includes detecting gestures by the corresponding individuals. 

26. The method of claim 22, wherein combining the captured media data 
includes detecting movements by the corresponding individuals. 

27. The method of claim 22, wherein combining the captured media data 
includes detecting locations of the corresponding individuals. 

28. The method of claim 22, further comprising refining the media data 
obtained from the sensor components in response to the respective activities. 

29. The method of claim 22, further comprising storing the combined 
media data in a history of each communication interaction. 

30. The method of claim 22, further comprising monitoring an artifact 
over time. 

31. The method of claim 30, further comprising recording a history of the 
artifact over time. 

32. The method of claim 22, further comprising detecting one or more 
activities in the environments and creating an interest area for each detected 
activity. 

33. The method of claim 32, further comprising associating the interest 
area with another interest thread. 
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34. A computer-readable storage media that contains a set of code that 
when executed provides communication by: 

providing a first set of sensing and rendering components for covering 
physical movements of multiple individuals present in a first environment; 

providing a second set of sensing and rendering components for 
covering physical movements of multiple individuals present in a second 
environment; 

detecting multiple communication interactions each involving a 
respective subset of the individuals present in the first and second 
environments; 

maintaining an interest thread for each detected communication 
interaction; 

capturing a set of media data from the sensing components; 

combining the captured media data in response to the respective 
activities of the respective subset of the individuals; 

communicating the combined media data to the rendering 
components. 

35. The computer-readable storage media of claim 34, further comprising 
selecting a respective subset of the sensing and rendering components for use 
for each interest thread. 

36. The computer-readable storage media of claim 34, wherein combining 
the captured media data includes detecting speech levels of the corresponding 
individuals. 

37. The computer-readable storage media of claim 34, wherein combining 
the captured media data includes detecting gestures by the corresponding 
individuals. 

38. The computer-readable storage media of claim 34, wherein combining 
the captured media data includes detecting movements by the corresponding 
individuals. 

Appeal Brief 17 

Application Ser. No.: 10/682,542 Attorney Docket No.: 200315123-1 



39. The computer-readable storage media of claim 34, wherein combining 
the captured media data includes detecting locations of the corresponding 
individuals. 

40. The computer-readable storage media of claim 34, further comprising 
refining the media data obtained from the sensor components in response to 
the respective activities. 

41. The computer-readable storage media of claim 34, further comprising 
storing the combined media data in a history of each communication 
interaction. 

42. The computer-readable storage media of claim 34, further comprising 
monitoring an artifact over time. 

43. The computer-readable storage media of claim 42,. further comprising 
recording a history of the artifact over time. 

44. The computer-readable storage media of claim 34, further comprising 
detecting one or more activities in the environments and creating an interest 
area for each detected activity. 

45. The computer-readable storage media of claim 44, further comprising 
associating the interest area with another interest thread. 
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EVIDENCE APPENDIX 

None. 
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RELATED PROCEEDINGS APPENDIX 

None. 
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